Distinguished Science Award

Every year scores of new pharmaceuticals enter the market, almost never with human fetal safety data. Such data typically accumulate during the "rst years of clinical use, in the form of case reports, case series, prospective and retrospective cohorts and case control studies. All of these methods su#er from serious sources of bias, o$en leading to alarming signals of teratogenicity that are later found to be false. !is review highlights major sources of bias, including the bias against the null hypothesis in its di#erent forms, ascertainment and recall bias, in fetal exposure to pharmaceutical molecules. DISTINGUISHED SCIENCE AWARD © 2010 CIM Clin Inv E349 Correspondence to: Gideon Koren, MD Hospital for Sick Children 555 University Ave Toronto, Ontario E-mail: [email protected] Clin Invest Med 2010; 33 (6): E349-E355. Because randomized controlled trials are very rarely carried out during pregnancy, and almost never during the "rst trimester of pregnancy, understanding the cumulative impact of bias is critical for evaluating teratogenic risk and for e#ectively counseling pregnant women exposed to pharmaceuticals. Since the thalidomide disaster, medicine is practiced as if every medicinal molecule is a potential human teratogen. While, in reality, very few drugs have been proven to adversely a#ect the human fetus, women and their health care providers hesitate or even refuse to prescribe or take pharmaceuticals during pregnancy sometimes even for life-threatening conditions [1]. Due to obvious ethical constraints, randomized control trials are almost never conducted during pregnancy. Even when control trials are conducted, their statistical power to show increased fetal risks is very limited; hence, the assessment of safety of drugs during pregnancy is almost entirely based on observational studies. Such studies, using either prospective or retrospective cohorts or using case control studies, involve collecting data from exposed and unexposed mothers (in cohort studies), or infants who have or have not su#ered a given adverse event (in case-control studies). !ese designs are subject to a large list of potential pitfalls; leaving these pitfalls unrecognized can lead to serious errors in interpretation, as will be documented in the next section. Acknowledging and addressing confounding factors in maternal fetal drug exposure is critical if one considers that every year hundreds of new pharmaceuticals enter the market. We will describe major sources of confounders and bias typical of observational studies. Major birth defects are relatively rare, accounting for only 1-3% of all births; hence large numbers of subjects are needed to show excess risk at alpha of 5% and beta of 80%. For example, 800 women are needed in a two arm study to show a doubling of cases of major malformations. !e situation is much more complex when trying to quantify one speci"c major malformation (rather than all malformations combined). For example, neural tube defects occur in 1:1000 births and valproic acid increases that risk to only 20:1000. !us, hundreds of cases of maternal exposure to valproic acid during the "rst 28 days post conception may be needed to document a statistically signi"cant increase from baseline risk. !e vast majority of cohort studies published to date are underpowered to show signi"cant di#erences, a problem acknowledged by many of their authors. A common way to overcome this issue is to combine similar studies into a metaanalysis, thus gaining a large sample size. As would be expected, lumping studies together into meta-analysis introduces its own set of issues, including heterogeneity among studies, inability to control for confounders, and under-reporting of negative studies (i.e., those showing no excess in adverse e#ects) [2]. Women taking medications during and a$er conception o$en su#er from conditions which may a#ect pregnancy outcome. For example, chronic hypertension during pregnancy is associated with high risk of premature birth, unrelated to which drug is used to treat the hypertension. We have recently shown that hypertensive women have greater than 20% rates of prematurity in their o#spring, whether treated with labetalol or methyldopa, in comparison with only 4% among healthy controls [3]. !is source of bias can o$en be corrected by having, in addition to the drug exposed group and healthy control groups, an additional group with the same medical condition but treated with another pharmaceutical treatment. Yet, even this solution may not always resolve the issue, as was acutely shown with the selective serotonine reuptake inhibitors (SSRI). During the last few years, several studies have suggested that paroxetine may be associated with an increased risk for cardiovascular malformation, and, in particular, ventricular septal defects. To address the potential “bias by indication”, several studies compared paroxetine to other SSRIs, still showing increased risk of cardiac malformations with paroxetine [4]. It has been claimed that other SSRIs do not cause malformations. Yet previous studies have shown that there is not another single example where one molecule causes malformations whereas another in the same class of pharmaceuticals does not ( retinoids, ACE inhibitors) [5]. What these studies did not point out is that, while all SSRIs are prescribed to treat depression, paroxetine has also been used to manage anxiety [6]. !ere is ample evidence that women su#ering from anxiety are more likely than those with depression to visit emergency rooms with their babies and to have diagnostic tests [6]. Hence, there is a higher chance that these babies will be diagnosed with a cardiac malformation. To complicate matters even further, most ventricular septal defects (VSDs) (the most common cardiovascular malformation) are of the muscular type, and tend to close spontaneously in infancy. If assessed later, whether in children exposed to other drugs or those unexposed, the likelihood of detecting VSDs further decreases. !is may create an ascertainment bias, wrongly suggesting that the “excess” in VSD is a direct teratological e#ect of paroxetine. Koren & Nikel. Sources of bias in signals of pharmaceutical safety in pregnancy © 2010 CIM Clin Inv E350 Prospective cohort studies take pride that by enrolling women in early pregnancy, before the outcome of pregnancy is known, they avoid the serious bias of retrospective data collection. Since most miscarriages occur during the "rst trimester of pregnancy, the later in gestation that pregnant women are recruited, the less likely they are to have a miscarriage. For example: if a group of women exposed to paroxetine is recruited at "ve weeks of gestation, whereas women exposed to other drugs are recruited at 10 weeks of gestation, clearly there is a greater chance that miscarriages would be detected and reported in the paroxetine cohort. !ere are two possible ways to overcome this type of bias: 1. By recruiting women in all arms of the study in the same week of gestation. 2. By post-hoc statistical adjustment for the time of enrollment. In summary, critical appraisal of studies correlating pharmaceutical exposure in pregnancy with rates of miscarriage must pay close attention to the gestational age at enrollment into the study. A large number of studies attempting to associate pharmaceutical exposure with malformation rates are retrospective in nature, that is, the cases were collected a!er pregnancy outcome was known. It is conceivable that women having malformed children would more likely report them to drug companies or to regulatory agencies. !is hypothesis was proven in 1999 by Bar Oz and colleagues, who compared a retrospective registry to a prospective one of the same drug (itraconazole) [7]. In the prospectively collected group, there was a 3% malformation rate suggesting no increased teratogenic risk. In contrast, there was a more than four-fold a (14%) malformation rate in the retrospectively ascertained cohort [7]. Acknowledging this serious source of bias is critical, yet there is also a “positive” message here: if, despite this bias, a retrospectively collected cohort does not exhibit a higher malformation rate than expected in the general population, it is conceivable that the drug is safe. Case-control studies typically enroll children with a speci"c malformation ( spina bi"da) and ask the mothers what pharmaceutical products they had used throughout pregnancy, and particularly during the "rst trimester of pregnancy. It has been argued that mothers of malformed children may have a di#erent pattern of recall than mothers of healthy children. Speci"cally, the malformation may facilitate memory in an attempt to "nd a pregnancy-related cause. !is source of bias can be remedied by collecting data from mothers of children with a di#erent malformation, unrelated to the hypothesis in question. For example, when Pastuszak et al. ascertained that Brazilian women giving birth to children with the Mebius sequence had a much higher likelihood of using misoprostol in an attempt to terminate pregnancy, than women giving birth to children with spina bi"da [8]. Recall bias in pregnancy is not limited only to remembering the use of pharmaceuticals during pregnancy, but also to recall of disease symptoms. We have shown in 200 women with nausea and vomiting during pregnancy, that when interviewed weeks to months a$er their symptoms had subsided, they reported signi"cantly worse symptoms than what they had described in real time [9]. A typical prospective cohort study during pregnancy recruits women exposed to pharmaceuticals before the outcome of pregnancy is known, and this is why the term “prospective” is used. At a later follow-up, women are asked about their health a$er the "rst interview and up till the followup interview. It is important to recognize that this part of the study is retrospective and hence open to recall bias. !is becomes very important in trying to correlate, for example, the severity of the disease with outcome. One of the most cited advantages of prescription databases linked to neonatal registries is that the dose of drug and length of treatment are not dependent on maternal recall. !e tradeo# is that prescription records do not prove that the pharmaceutical was actually taken by the pregnant mother, only that they were prescribed. !e seriousness of this source of error was acutely exhibited when Jick and colleagues correlated maternal prescription of spermicides with congenital malformation [10]. It was argued that prescription of spermicides before conception did not yet mean that the women took them. Indeed, in a follow-up study of the malformed cases in this study, Watkins showed that almost none of the mothers took spermicides only following conception [11]. Many administrative database studies do not have data on "ndings among women who elected to have an abortion; rather, they report of “liveborn infants”. Levy and colleagues hypothesized that a signi"cant number of elective abortions are performed due to a major malformation diagnosed . Without such data, researchers take the risk of missing a signal. Koren & Nikel. Sources of bias in signals of pharmaceutical safety in pregnancy © 2010 CIM Clin Inv E351 Using the example of antifolates in pregnancy, known to increase the risk of neural tube defects, Levy et al. showed that in their cohort, such an association was apparent only when elective abortion data were considered too, but not when only live births were counted [12]. Alcohol, maternal smoking and other drugs of abuse may increase fetal risks, including: birth defects, prematurity, intrauterine growth retardation, miscarriage, still birth and developmental teratogenicity. Due to shame, guilt and fears of losing custody of a child, it is conceivable that women seeing adverse e#ects in their o#spring may underreport on drugs and alcohol abuse. !is hypothesis was proven by Wong studying a cohort of women who were counseled by a teratology information service during the "rst trimester of pregnancy. In particular, their reports on cigarette smoking were probed. When reinterviewed a$er the birth of the child, women who had healthy babies reported a similar patterns of smoking to that which they had reported initially. In contrast, women giving birth to o#spring experiencing adverse outcome tended to minimize the numbers of cigarettes consumed when compared to their original reports [13]. !is type of bias may seriously impair studies on the e#ect of alcohol, cigarettes and drugs of abuse by introducing misclassi"cation, i.e., women who smoked are erroneously classi"ed as non-smokers. Bias against the null hypothesis occurs when “positive” studies (e.g., studies showing a drug to be teratogenic) are more likely to be submitted for scienti"c meetings and journals, to be presented, published and publicized than “negative” studies (e.g., those suggesting the pharmaceutical is safe). !e seriousness and pervasiveness of this bias has been shown at each step of the act of reporting results. Investigating the fate of studies submitted to the Oxford University ethics committee, Easterbrook and colleagues have shown that “negative” studies were signi"cantly less likely to be submitted or published in peer reviewed journals. Of interest, this was due not only to higher rates of rejection of the papers by the journals [14], but also due to the perception of the investigators that their studies were less likely to be accepted. More related to adverse pregnancy outcome, we have shown that abstracts failing to identify reproductive risks of cocaine were less likely to be accepted by the Society for Pediatric Research than “positive” studies, despite the better quality of the negative studies [15]. !e bias against the null is further augmented by the lay media that tends to publicize signi"cantly more “positive” studies than “negative” ones. !is grim reality was documented unequivocally in the case of two studies published back to back by the Journal of the American Medical Association ( JAMA). In 1992, JAMA published two studies dealing with the risk of radioactive exposure [15]. A study on the fate of several thousand workers in Oakridge developing the American Atomic bomb in the 1940s has shown increased risk of leukemia. In contrast, a study investigating whether residing near nuclear energy plants failed to show more cases of cancer. Despite similar exposure in the journal, the “positive” study was cited by the lay media signi"cantly more o$en [16]. It is now evident that the bias against the null hypothesis is pervasive and encompasses every step of the production of new knowledge, starting with the authors believing that they had a lower chance of publishing, continuing with lower probability of selection for presentation at medical meetings, and continuing on to lower rate of publication in science journals. Because of the con"dential nature of the editorial process for selecting peer review papers for publication, it is impossible to verify whether the editorial process also results in bias against the null hypothesis. !e last chain of events in the process of distribution of new medical knowledge related to pharmaceutical risk in pregnancy is its citation by medical scientists in subsequent papers. If “positive” studies are cited more o$en than “negative” studies, then the risk of biased and misleading data is entrenched at the highest level. For the purpose of this review we wished to verify whether this type of bias against the null hypothesis exists. Over the last "ve decades, a large number of medications were initally implicated as human teratogens only to have these conclusions refuted later by larger numbers of negative studies and meta-analyses. !is has raised serious concerns regarding bias against the null hypothesis: where negative studies ( not showing adverse fetal e#ects) being less likely to be submitted for publication by their authors [17] less likely to be accepted to scienti"c meetings, or be reported by the lay media. !e role and impact of medical journals themselves on this type of bias have not been studied. Several recent cases, where major medical journals published initial reports suggesting a drug to be teratogenic, only to be subsequently refuted by numerous other studies, have prompted us to examine determinants of scienti"c citation. Koren & Nikel. Sources of bias in signals of pharmaceutical safety in pregnancy © 2010 CIM Clin Inv E352 !e focus of our interest was the number of citations that “positive” ( showing adverse fetal events) “negative” scienti"c articles have accumulated. We surmised that cumulative citation numbers re&ect the e#ectiveness of knowledge transfer and determines on what health care providers are basing their clinical decisions on. For the purpose of this review, we selected six drugs which, over the last "ve decades, have been the focus of appreciable controversy over their teratogenic potential in humans. In all selected drugs, initial reports in major medical journals implicated the drug as a human teratogen, but these claims have been later refuted by large numbers of negative trials. 1. !e oral contraceptive pill, reported in the 1960s as causing sexual malformations [18]. Subsequently, a large number of studies and two separate meta-analyses refuted this claim [19]. Despite this, the “pill” was designated a “category X” by the FDA, a label that was corrected only recently. 2. !e antinauseant Bendectin® (doxylamine plus pyridoxine) was used by up to 40% of American women in the late 1970s for morning sickness. Several highly publicized reports and legal cases resulted in removal of the drug by its manufacturer from the American market in 1983 [1], despite numerous “negative” studies and three metaanalyses showing its apparent safety [2]. !e drug has been continually used in Canada and is being introduced again in the US. A$er its removal, leaving American women without an FDA-approved drug for morning sickness, the rate of hospitalization of pregnant women for severe vomiting more than tripled [20]. 3. Benzodiazepines are widely used by women of reproductive age. Because half of all pregnancies are unplanned, large numbers of women unknowingly exposed their fetuses to this class of medications. A highly publicized study in the 1980s caused tremendous concern [21], despite numerous negative studies [22]. On a molecular level, it has been argued that the GABA receptor antagonist properties of benzodiazepines may explain their teratological e#ects [23]. 4. !e selective serotonin reuptake inhibitor, paroxetine, has been the leading SSRI during the early 2000s. Preliminary, highly advertised reports claimed that the drug causes cardiac malformation, leading the FDA and Health Canada to issue warnings. !ese warnings were not reversed, despite large numbers of studies and a metaanalysis [4] refuting these preliminary claims, and causing large numbers of women not to treat even life-threatening depression [24]. 5. !e cholesterol synthetase inhibitors statins have been implicated as human teratogens based on uncontrolled case series in a highly publicized paper [25]. Several later papers and a systematic review refuted this claim [26]. !e statins inhibit the synthesis of cholesterol, which is critical for fetal brain development. Indeed, animal studies have shown high rates of adverse central nervous system e#ects [27]. 6. A highly publicized paper claimed that the antihypertensive ACE inhibitors cause congenital malformations [28]. !is class of drugs is typically discontinued as soon as pregnancy is recognized, because it has been proven to cause fetal renal damage and hypocalvaria in late pregnancy, yet the researchers claimed that the ACE inhibitors caused "rst trimester malformation. Consequently, large numbers of women with pre-pregnancy hypertension were advised that the use of ACE, before they recognized that they had conceived, may have caused fetal malformations. Importantly, all other studies on this topic to date have failed to show increased teratogenic risk [2931]. We reviewed all papers included in the systematic reviews and meta-analyses of the six selected drugs (oral contraceptives, bendectin, benzodiazepines, paroxetine statins and ACE inhibitors). Papers were classi"ed as “positive” if the primary endpoint (rates of malformations) was signi"cant at p<0.05 in comparison with the control (unexposed) group. Papers were classi"ed as “negative” if the rates of malformation in the exposed group were not signi"cantly higher than in the comparison group. !e following characteristics were identi"ed for each study: a. Year of publication. b. !e impact factor of the journal in the year of publication as reported in ISI Web of Knowledge Journal Citation Reports. c. !e total number of scienti"c citations of the study. !e number of citations to each study was retrieved through a cited reference search in Web of Science In analyzing the data, we "rst compared the numbers of citation of “positive” “negative” studies using the Mann Whitney U test. Subsequently, we conducted multivariate linear regression analysis with the total number of citations per paper as the dependent variable, and the journal citation impact, year of publication and being “negative” or “positive” as independent variables. !is analysis aimed at identifying determinants that predict the total number of citations of a paper. Koren & Nikel. Sources of bias in signals of pharmaceutical safety in pregnancy © 2010 CIM Clin Inv E353